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ABSTRACT 

An important objective of the History Academy for 
Ohio Teachers was to shift the participating teachers 1 beliefs and 
orientation to history. Through the examination of the portfolios and 
interviews, the study looked at the changes from the participants' 
perspective, A subset of 12 teachers were selected from the Academy's 
participants and were interviewed during portfolio conferences at the 
end of the 4-week summer session, and a second interview was held 
between February and April of the next year. Emerging patterns were 
discussed, and a coding system was developed and revised. The results 
indicated that overall the Academy appeared to achieve its purpose of 
creating a momentum for history education and affected changes in 
teachers' views of themselves, their knowledge, and their practice. 
The Academy successfully customized its approach to meet the diverse 
backgrounds, needs, assumptions, and individual differences of the 
participants. The findings suggested that the portfolio evaluation 
produced more trustworthy and richer data than traditional means of 
assessment and that the Academy was successful in its attempt to 
change participants' perspectives about the discipline of history and 
about themselves as teachers. Appendix A provides a summary of key 
terms and elements, and appendix B describes a sample case study of a 
part icipant . (CK) 
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INTRODUCTION 

If doing history requires the careful examination of primary sources from multiple 
perspectives, then doing an evaluation of histories should not be based upon any lesser standards. 
The present report is written with three purposes in mind: first, to present procedures for 
evaluating the history of teachers consistent with the standards expected of doing history; second, 
to present the results of an evaluation of an academy intended to improve history teaching; third, to 
invite interested parties to invest in reviewing the lengthier report upon which all of this is based. 

As a report of evaluation procedures, this report is unique on a number of fronts. It reflects 
the use of portfolio evaluation procedures as a way of pursuing a grounded evaluation of the 
influence of a project upon the intellectual jo :rney and practice of teachers. The data emerging 
from these portfolios are complex and varied providing a rich and detailed picture of individual 
teacher's changes, as well as the factors or elements which contributed to those changes. Without 
compromising the integrity of the assessment, these data and data collection procedures were 
pursued collaboratively and examined on their own terms. In general, these data seem more 
trustworthy and richer than more traditional means of assessment. Rather than rely upon 
secondary data, these data appear primary; instead of pursuing objectives and uniform 
interpretations based upon standardized sampling procedures, these data have been clearly 
grounded in events, experiences and over time. Moreover, these data are treated on their own 
terms en route to affording a complex assessment across individuals using graphic ways to depict 
and quantify data without compromising the nature of these data. 

As a report of the effectiveness of the Ohio History Academy, this report should not be 
viewed as a description of the Academy but as a report of the Academy's effectiveness - especially 
the different elements. For a more detailed description of the Academy see Doing History: A 
Model for Helping Teachers, 1993, National Council for History Education, Westlake, Ohio. 
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Background on the Academy and Evaluation Plan 

The History Academy for Ohio Teachers had as its goals: 

• to bring classroom teachers to a firmer grasp of the discipline of history, of 
changes in history instruction, and of the most effective methods of teaching 
history (including technology); 

• to help teachers feel confident of their own abilities to pursue further self- 
education; 

• to enable teachers to act as mentors to their colleagues; 

• to enable teachers to become equal participants with university scholars in 
collaborative efforts. 

Key features of the Academy were: 

• An integrated curriculum which presented courses in World and United States 
history, as well as courses in historical theory id methodology consistent with 
the report by the Bradley Commission; 

• An emphasis upon certain historical processes such as the use of primary sources 
and. multiple perspective-taking; 

• Participants drawn from elementary, middle school, and secondary classrooms; 

• First-hand experiences with nationally known historians throughout the 
Academy and in conjunction with a mentoring program involving historians 
throughout Ohio; 

• The exploration of a rich array of topics, teaching ideas, resources, and 
alternative ways of engaging with these ideas; 

• An intensive residential experience across four weeks in the summer, 

• Follow-up on site with the teachers as they pursued changes in practice tied to 
shifts in their thinking over the summer, 

• Outcome products combined research (research papers) and practice (teaching 
units). Opportunities for teachers to develop customized teaching units 
pertaining to specific topics or historical periods. Other tasks, included writing 
personal journals, and the maintenance of personal portfolios. 

The structure of the Academy curriculum and these tasks were intended to do more than 

provide teachers with more information about their subject content One important objective of the 

Academy was to shift the teachers 1 beliefs and orientation to history, their practices of teaching 

history, and their views about themselves as historians and teachers. From an evaluation 

standpoint, three questions help frame the assessment: 
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• What kinds of shifts in beliefs and values resulted from teachers' participation 
in the History Academy? 

• How has the academy experience affected the participants in terms of their 
classroom practice, their relationships with colleagues, and their general 
professional decision-making? 

• Which aspects of the History Academy can be linked to these shifts? 

These issues were researched through procedures which were intended to document and 
recover the participants 1 reflections about the Academy experience. To this end, teachers were 
engaged in a process which relied upon a grounded data collection and analysis using portfolios 
and interviews as the primary vehicles. This approach seemed appropriate for several reasons. 

We wanted the participants to share in the evaluation procedures. For example, staff and 
teachers worked in partnership with the evaluation team to develop and implement overall 
procedures, co-develop data collection techniques, and collaborate in the interpretation of results. 
We feel this approach increased the likelihood that we weie fully informed with respect to the 
experience of the staff and participant rather than operating solely on a priori notions about 
effectiveness and outcomes. In addition, by engaging participants in this assessment, we expected 
that these procedures would eventually be adopted as on-going, self-evaluation. 

We wanted to ground the evaluation in the actual experiences of the participants. That is, 
we wanted to determine the effects of the Academy by examining direct evidence of such effects 
(e.g. artifacts from the Academy participants 1 portfolios, projects, etc.). We also wanted to be 
able to codify the shifts in teacher beliefs, attitudes and actions and the elements which contribute 
to such shifts. We wanted an approach that would sensitive to such shifts revealed from the 
participants' perspective. 

Finally, consistent with the Academy's aim to have an effect on classroom practice we 
wanted to present teachers with a model of alternative evaluation. Teachers were informed of 
portfolio techniques and encouraged to explore alternative assessment techniques in their own 
classrooms. 
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In addition, the procedures used in this evaluation were intended to be consonant with the 
dominant themes, standards, and approaches of historical investigations. In this sense, both 
investigations are parallel in that: 

• Accounts of events are grounded in primary rather than secondary sources; 

• Interpretations are explained, as well as grounded in the primary source 
material; 

• Interpretations are always put in the context of the sources used and what these 
sources are purported to represent; 

• Primary source material is sought which allows an examination of patterns over 
time, in different settings, and from different perspectives; 

• The primary source material is not predefined or limited, nor is the same 
material expected to be viewed the same way by all persons; 

• The emerging accounts may be revisited, revised or extended based upon 
additional evidence or by applying a different perspective to the data; 

• The interpretations themselves remain open to different interpretations. 

The evaluation plan which emerged not only assessed learning and evaluated the role of 
certain elements, it contributed to the support of some of the key elements of the History 
Academy's program. Attempts to highlight these parallels were frequent. Through formal and 
informal discussions with Academy participants and staff explicit observations were made 
regarding the fit between the instructional plan and the evaluation plan. 

Procedures 

The major data source for the evaluation were portfolio-based interviews conducted at the 
end of the summer workshop and then during then again during the school year. More 
specifically, teachers who participated in the History Academy were responsible for creating and 
maintaining personal portfolios The purpose of the portfolio, as originally presented, was to serve 
as a mechanism for evaluating the teachers 1 development as historians and as history educators. 
Additionally, the evaluation team used these portfolios as the primary vehicles for assessing the 
impact of the Academy on the participants, that is, how features at the academy contributed to 
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changes in teachers 1 knowledge and practice as well as how the Academy and its various elements 
were viewed. 

At the end of the four- week summer session, teachers were scheduled for individual 
portfolio conferences during which an Academy faculty member "debriefed" the participant 
regarding his or her Academy experience. A member of the evaluation team was also present to 
record the interview using handwritten field notes, audio tapes and video. 

A subset of twelve teachers were selected for intensive study. In general, the 
characteristics of the twelve teachers in the study group possessed a range of educational 
backgrounds similar to the general group. Teachers in the subset had each agreed to have their 
interviews either audio taped or video taped. A second set of interviews took place in February, 
March, and April of the follow-up year. The follow-up conferences we*e similar to the summer 
interviews. Participants were given an advance organizer as a way to prepare for the interview. 
Teachers were again encouraged to use their portfolios to represent experiences which had 
occurred since the summer session. Teachers in the subsample were again the subjects of intense 
study which utilized the same procedures as the summer interviews. The follow up interviews 
differed from the summer interviews in thai hey were held at the teacher's school, members of the 
Academy faculty were not present during the interviews, and the interviews were conducted by 
members of the evaluation team. Field notes, audio tape and videos were made of the interviews. 

Following the interviews, the members of the evaluation team met to discuss emergent 
patterns observed and apparent from reviews of the tapes and transcripts made of interviews. Over 
several re sw sessions several renditions of a coding system emerged and were revised. The 
coding system was applied across the two interviews, perspectives (historian, teacher, individual), 
in terms of impact (resources, teaching strategies, curriculum issues, content knowledge base, 
historical processes, collegiality, and technology), shifts in knowledge (new ideas, shift in beliefs, 
reinforcement of beliefs, and no effect), and levels of utility (rejection, citation, understanding, 
appropriating, and imparting). Figure 1 depicts the coding system in terms of the broad categories 
and the overall relationship to one another. 
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A3 EVIDENCED THROUGH 



The data acquired from these interviews and portfolio artifacts amassed by each teacher 
were recorded and these transcribed records served as the basis for the development and 
implementation of analysis procedures. Below (see Figure 2) is a portion of a teacher's transcribed 
response to our interview. In the first two columns are the codes that were assigned this portion. 
These codes coincide with the aforementioned layers-perspectives (H for history, T for teaching, I 
for Individual), level of use (1,2,3,4,5), shifts in understanding (n for new knowledge, etc.) and 
the specific elements which was connected with these features. 







AGM: 


Have your students' approach to history changed as a result of 
your academy experience? 


07a 
07b 
07c 
07d 


H/4/S/TS/C01 
T/4/S/TS/C01 
H/4/S/HP/C01 
T/4/S/HP/C01 


Rachael: 


You saw them there. They love Ohio history. They really do. I 
think I have a real enthusiasm for Ohio history because it is my 
favorite but I think they like it and they're seeing - 1 find them 
ccmlng to me and saying things like now is that a primary 
source? And it's really funny. For a while there I used that word 
for the first several times I used it and then I would come back to 
it like the day or something and I would say what was that word I 
was always using? What would this be called? And they 
couldnl quite remember it. And now they're remembering it and 
saying it. Yes, 1 think they really seem to enjoy it more. I'm 
hoping - we had a speaker In yesterday from the Ohio Historical 
society on Ohio In the Civil War. And a little bit above where 
we're at right now. But he came in in full dress and one of the 
kids asked - 1 was so proud of him - they said are those 
authentic? Where did you get your information for. It was cute. 
So 1 think yes. They're approaching it (inaudible). 



Figure 2 



Teachers generally grounded their responses or presentations by referring to specific items 
in their portfolios. These items became the focus for parsing the interview into topical blocks. In 
fact, teachers had been instructed in ways to cluster or stack artifacts into topical categories. The 
parsing procedure continued until the entire transcript was segmented. 
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Next, at the level of specific blocks, the participant's interpretation was analyzed using a 
method of constant comparison to describe and label the characteristic features of each reference* 
As the analysis continued through several participants, coding categories emerged, developed, 
expanded and collapsed until some general patterns appeared. This ultimately led to a system in 
which each reference a participant made during the course of the interview was given four different 
coding decisions. While most descriptors were mutually exclusive, some coding decisions 
attributed multiple descriptors to a single reference. This coding procedure and the descriptive 
categories which follow were developed in collaboration with the Academy director and staff. 

Because the portfolio conferences were constructed to encourage participants to cite 
evidence of their experiences, references tended to be fairly specific and easily identified. The 
interview transcripts were coded until we felt comfortable organizing responses into meaningful 
clusters. We view these clusters as categorical and not hierarchical. 

Topics 

On a first pass through the transcripts responses generally could be grouped in one or more 
of the following topic categories: 
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RESOURCES 



TEACHING 
STRATEGIES 



CURRICULUM 



These tended to be sources of information about a variety of pedagogical 
and conceptual issues and contained references to printed matter, including 
textbooks provided by the Academy, content specific materials, lecture 
notes, and handouts; alacm including the Ohio Historical Society, the Ohio 
State library, Ohio Arts Council; and p eople, including mentor teachers, 
professional historians, and colleagues. 

These tended to be references to specific classroom activities or ways of 
organizing the classroom such as involving students in research projects 
using primary source documents, taking field trips, dramatizations, read 
aloud, and writing a personal journal. 

These tended to be references to larger issues of pedagogy, scope and 
sequence of instruction, educational policy, and the focus of instruction at 
particular grade levels or on specific guidelines or texts which shape what is 
taught. For example, one teacher (Olivia) commented, "I tagged the list of 
participants who are putting together the state model curriculum, because 
they're obviously going to have a profound impact on what I'm doing In the 
classroom." 



CONTENT 

KNOWLEDGE 

BASE 



HISTORICAL 
PROCESSES 



These tended to be references to specific information pertaining to the 
teacher's research topic and/or model unit. For example, a teacher (Wanda) 
interested in the Underground Railroad stated, "Talking about white 
abolitionists, I also learned that Quakers as a whole, were not anti-slavery. 
They were very pacifist. They owned slaves but didnl go and seek out 
fugitives." 

This term describes the concepts and issues which reflect an historian's 
approach to learning and teaching history and includes references to critical 
evaluation, influence of culture, multiple perspectives, the post-hole 
approach, and primary source documents. 



COLLEGI ALITY These were references to interactions with the Academy personnel and other 
Academy participants, other teachers in their schools, and with other 
teachers through professional meetings or courses. 



TECHNOLOGY 



These were references to the computer software demonstration, the 
computer-aided library research system, and the use of word processors to 
prepare research projects. 



Perspectives 

A second pattern to emerge from the analyses was that participants could not interpret the 
significance of a piece of evidence without adopting a particular perspective . The evaluation team 
may have helped shape the process of "making visible" such perspective-taking by modeling an 
organizational procedure. In general, then, participants tended to be explicit about the perspectives 
they were using as they interpreted each reference. Participants fluctuated in the perspectives they 
adopted, however three perspectives were clearly dominant — namely their development as 
historians , as teachers , and as individuals . 
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These perspectives, wliile distinct from each other, were not mutually-exclusive. That is, a 
teacher might respond to a particular portfolio item or an element of the Academy experience from 
the perspective of a teacher and from the perspective of an historian. For example, one teacher 
(Harriet) commented about a classroom practice which reflects both an historian and a teacher 
perspective. "Sometimes I'll let them debate or argue or try to come to a conclusion about 
something that happened in history and I try to let them know that everything you read is not 
always actually what happened. For instance, when we write in our journals, we're writing about 
the same experiences but everyone has a little bit different attack on it or approach to what we do in 
school so they seem to understand that" 

Shifts in Understanding 

A fundamental concern was the ability of the program to shift teachers' understandings and 
attitudes about the discipline of history, themselves as historians and as teachers of history. On a 
third pass through the transcripts, references were analyzed based on occurrence and direction of 
those shifts. Here again, the labels used to code these shifts were drawn directly from the 
interpretations and comments participants used to describe them. These labels quickly merged into 
four distinct categories: New knowledge , shift in knowledge, and reinforcement Qf knowlsdse. 
and rejection . These categories were mutually-exclusive and were generally explicitly stated by the 
participants. For example, Denise commenting about the topics contained in her model unit 
observed, "You could get history in all these. I think that's something I would not have done 
before this Academy. I would not have looked at how history affects literature and how history 
affects geography. I think that's a definite area as a teacher that I've grown." Other participants 
noted that some aspects of the Academy reinforced their own beliefs and practice. For example, 
Steve noted, "I know that some of the elementary teachers said they never heard of them, but, you 
know, course working with high school kids I've used primary sources to some extent. Again, 
this reinforced what I was doing but also I can see how much more I can use it." Some 
participants noticed that the Academy experience caused them to reconsider some ideas. For 
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example, Paige commented on the impact of discussions about primary source documents and the 
presence of bias on the way she developed a teaching unit on the Holocaust. "One of my plans 
was to do a day in the life of different people in the Holocaust because I had a lot of those people 
on tape. So I was going to use that tape as my primary source. And then when I started thinking 
about it, one of the reasons I changed was I realized how probably biased — I ! m sure it was 
biased, their reactions were." Finally, some teachers rejected particular features simply 
commenting that the features had no value or application. 



Intensity/Utility of Experience 

A fourth pass through the transcript attempted to focus attention on the intensity/utility of 
the experience . That is, interpretations were compared for the purpose of determining how useful 
an element was deemed to be or how strong a particular shift seemed to be. In general, 
participants 1 responses appeared to evidence four critical landmarks. These landmarks were given 
the codes of "citing", "understanding", "appropriating", "imparting", or "no effect". 

References were coded as " citing " generally when they were list-like and unelaborated. 
Rachael, for example, described artifacts in her portfolio in the following manner, 'This one has 
been a great book too and it's called an Ohio Reader. And this is a newspaper article from the 
Heritage, which is a Mid-Ohio valley newspaper. And this is something from the Marietta Times." 
While these items pertain to her model unit and research project, it isn't clear how the materials 
might be used. 

References were coded as " understanding " when the teacher's comments reflected any 
depth of knowledge about the item or artifact. Most often, these tended to be bits of topic specific 
content information. However, Gloria noticed an increase in understanding of historical concepts 
after reading Building a History Curriculum. "When we got here we got this nice easy to carry 
version of the Bradley Commission and I started using it and so the History Academy has made me 
more aware of the organizational ways of doing this. I taught the same things but it kind of gave 
you a condensed way to look at it" 

» 13 
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As teachers began to consider ways they might use the concepts and content acquired 
through the Academy experience, their references were coded as "a ppropriation ". For example, 
Steve commented about a dramatization strategy modeled by one of the mentor teachers. "And so I 
came down here wondering if that was an area th?,i 1 might get some more information on Well 
then Nancy Taylor, she does this at the Ohio Village. Soon as I heard about that I got to thinking 
that i ought to ask her, you know, what she does. And I guess I came down here thinking that in 
the uh — when they do those characters I think it's got to be someone like Will Rogers or 
Abraham Lincoln or Franklin Delano Roosevelt I never, I don't know why, but I never thought 
of creating a character. And that just hit me. You know, to just personify a person, a fictional 
person from the time period, build it from a primary source So I asked her about it and she gave 
me outlines and examples of ways that it's done at Ohio Village and other people she knows." 

Finally, during the school year teachers found ways to implement or "impart" the concepts 
as well as the content they developed through the History Academy, Donald, for example, adapted 
the notion of a personal portfolio to meet the needs of his world history class. "In the world 
history class I used to do what I called the personal history project. I've always had the students 
do a genealogy of their family and try to collect in order to write a paper on their own personal 
history. But I got to thinking after this academy that it might be interesting to try developing kind 
of a portfolio package. And so we have portfolio days in history class and we're talking, for 
example, this Friday will be a portfolio day and we'll ask each student to tell how far they've gone 
on which parts of their project and put together an assessment sheet where we can each time we 
interview the students we can talk about where are you now, how far have you gone, and so 
forth." 
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Unit of Analysis 

For purposes of coding transcripts, each interview was separated into idea units. (Refer to 
Figure 1.) A single idea corresponded to the discussion of a topic in terms of its perspective, 
impact, and source. If a topic were subjected to a discussion which extended to other sources or 
other impacts, or a change in perspective, then more than erne coding would be applied. With each 
new topic new codings were also pursued. 

Reliability 

Meetings were held almost weekly during the months of development and implementation 
of coding procedures. During preliminary coding development, evaluators examined the interview 
data separately before meeting to discuss findings. Similar coding categories were found by 
individual evaluators leaving only a common designation to be determined for those codes. When 
a common coding scheme had been agreed upon, codes were applied by two independent raters to 
the transcribed Summer Interviews. Upon discussion of the procedure, the raters independently 
arrived at almost identical findings. One interview was compared line by line with over 90% 
agreement in assigned codes. Several other Summer Interviews were spot checked with similar 
agreement. When all twenty-four interviews were transcribed and coded a second rater checked 
and verified codes with the same percentage of agreement. 

Graphic Representations 

For each of the twelve participants in the close-up group, the responses were then 
reorganized by categories and perspectives, and incremented by direction and intensity. Each 
response was then assembled to represent a visual tally. These visual tallies help reveal important 
themes of the Academy experience. 

For each individual, patterns emerged regarding the types of items presented in the 
portfolio as well as the perspectives the participant was able to adopt. In this way, an image began 
to form in terms of the ideas, strategies, and resources participants were taking away from this 
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experience and how they viewed those ideas, strategies, and resources. Three graphics - 
Historian, Teacher, and Individual- were constructed for each teacher in our in-depth study group. 
Within each graphic (Historian, Teacher, Individual) the categories of the items to which the 
participant refers, (i.e., Resources, Teaching Strategies, Historical Processes, etc.) are identified. 

The visual representation helps identify the intensity/utility of the Academy experience 
overall, as well as the nature of the ideas. Examining these visual representations, patterns emerge 
which suggest how individuals acquire and integrate certain ideas or strategies through the 
Academy experience. Also, the visual representation also permits the identification of key elements 
which tended to facilitate knowledge shifts. After analyzing the transcripts a list of specific 
elements, events, and concepts emerged. These specific features were designated "hot spots". 
Below is the graph (see Figure 3) for Brenda as Historian at the time of her Summer Interview and 
immediately following in Figure 4 is the graph as Historian at the time of her Follow-Up 
Interview. 

In the Summer Interview graph, Brenda has three codable statements from the perspective 
of Historian. Under Resources from the perspective of Historian, she has commented at the level 
of Understanding regarding New Knowledge about Primary Resources (C01). There were no 
comments applicable to Teaching Strategies or Curriculum. Under Knowledge Base from the 
perspective of Historian she has commented at the level of Understanding regarding a Shift in 
Knowledge. She has also made reference to Historical Processes at the level of Understanding 
regarding a Reinforcement of her own knowledge respective to Primary Sources (C01). 
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At the time of her Follow-Up Interview, Brenda had twelve codable comments from the 
perspective of Historian. Under Resources she has three codable items which are at the level of 
Appropriation which is indicative of her movement from Understanding to actually using 
information on Primary Resources (C01), Building a History Curriculum (R01), and Critical 
Evaluation (C02). The first two codings are expressed as New Knowledge and the statement 
regarding Critical Evaluation (C02) as a Reinforcement of her prior knowledge. 

Again there are no codable items under Teaching Strategies, Perhaps as a result of having 
been in the classroom she now has four comments regarding Curriculum, One regarding New 
Knowledge, one regarding Reinforcement of prior knowledge, and two indicating a Shift in 
Knowledge. There were no codable comments from the perspective of Historian regarding 
Knowledge Base or Technology. Under Collegiality she has made a comment indicating that there 
was No Effect at the level of Rejection regarding her Historian (R13). Generally, comments which 
were coded like this indicated that the Historian was not a particularly useful aspect of the Academy 
for the participant for reasons such as the Historian was not very accessible or the Historian was 
not especially helpful, in which case, the participant Rejected further pursuit of possible Resource 
of Historian (R13). 
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Figure 4 



Similar comparisons for Brenda can be made with the charts from the perspectives of 
Teacher and Individual. Graphic representations for all of the teachers are included in the fuller 
report In the Appendix is a detailed listing of the codes used and a single sample case study. 

In an attempt to summarize the data acquired from the twelve case studies, several analyses 
were pursued in which we pursued the following: 1) graphically displayed the shifts in 
understanding and the utility of the academy for each of the teacher, and 2) collapsed the data 
across the subsample of twelve teachers by the following: Interview (summer or final follow-up); 
Perspective (teacher, historian, individual); Shift in understanding (new knowledge, shift in 
knowledge, reinforcement of knowledge, or no effect); intensity of experience (citation, 
understanding, appropriation, imparting, or rejection). 
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FINDINGS 

From an examination of the data from each teacher's portfolio it was apparent that across all 
of the teachers, an array of changes occurred in their knowledge and practice over the summer 
workshop and into the school year. Moreover, these changes occurred in terms of shifts in their 
understandings, as well as new ideas which were appropriated, used and imparted to others. 
Most notable were the changes which occurred in terms of understandings and the extent to which 
ideas were appropriated as well as improved Each of the teachers changed in terms of themselves 
as historians, teachers, or individuals, relative to knowledge shifts and the extent to which ideas 
were used. Predictably, it should be noted that the teacher perspective was dominant — in 
particular, the participants were about twice as likely to view and evaluate an experience in terms of 
its classroom application as they were in terms of its relationship to themselves as historians or in 
relationship to their own personal interests. Along these same lines, while the participants 
generally appeared to understand ideas they had acquired through the Academy experience, in the 
"teacher" perspective, they were more likely to act on these ideas, appropriate these experiences, 
and adapt them to suit their needs. 

Participants make frequent references to how ideas gleaned from the Academy were new. 
For example, participants were about five times as likely to refer to an item as "new 1 ' as they were 
to view it as a "change" or as "reinforcement". 

Tables 1-2 represent a summary of the impact of the Academy across all twelve teachers 
based upon a compilation of the data from each teacher's portfolio. 

Table 1: Utility of Experience By Perspective for Twelve Case Study 
Academy Participants During Summer Interview 





Citation 
. 0) 


SUMMER 

Understanding 

0 


INTERVIEW 

Appropriating 

._Q) 


Imparting 
® 


Rejection 
© 


Teacher 


138 


160 


98 


0 


1 


Historian 


46 


94 


70 


0 


1 


Individual 


51 


91 


31 


0 


1 
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Table 2: General Knowledge Shifts by Perspective for Twelve Case 
Study Academy Participants During Summer Interview 





New 
Knowledge 
fcfl 


SUMMER INTERVIEW 

Shift in Reinforcement 
Knowledge of Knowledge 

fR\ (PC\ 


No 
Effect 
U 


Teacher 


271 


42 


30 


4 


Historian 


143 


38 


29 


1 


Individual 


114 


35 


23 


2 



These data indicate that different Academy participants found different elements of the Academy 
more or less useful and significant (i.e., there was considerable variation across participants in 
terms of which elements contributed to changes in knowledge and usefulness). However, there 
were a number of elements to which different participants made repeated references. The 
following (referred to as "Hot Spots") are among the twenty-nine elements which comprised the 
list of critical items to which repeated references were made: 



RESOURCES 

R01 Building a History Curriculum 

R02 History Matters 

R03 American History Textbook 

R04 World History Textbook 

R05 After the Fact 

R06 Historical Literacy 

R07 Narrative of Frederick Douglass 

R08 Lecture notes 

R09 Model units - Betty, Nancy, at al. 

R10 Handouts 

R1 1 Topic specific printed matter 
R12 Topic specific other media 
R13 Historian 
R14 Portfolios 
R15 Journal 

R16 World history course 
R17 American history course 
R18 Theory and methods course 



EVENTS 

E01 Guest lectures 

E02 Ohio Historical Society 

E03 Ohio State Library 

E04 Computer Demonstration 

E05 Grade Level Small Group 

E06 Discussion Group 

CONCEPTS 

C01 Primary Sources 
C02 Critical Evaluation 
C03 Multicultural Perspective 
C04 Posthole Approach 
COS Significant Individual 



Over half the participants 1 responses referred to these elements. 

Tables 3 and 4 depict these data in more detail (by perspectives, shifts in understanding and 



utility). 
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Table 3: The Relation of Utilization of 

During Summer Interview 
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Table 4: The Relation of Knowledge Shift to Critical Items by 
Perspective During Summer Interview 
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As illustrated in Table 4, in addition to differing frequencies of response, teachers also 
reported different levels of utilization for the each of the critical items. About a third of the 
responses (159) referred to print (e.g., library books, newspapers, journals) and nonprint (e.g., 
photographs, maps, realia) materials teachers had collected as resources for the research papers 
they were writing or for the teaching units they were developing. Other features of the Academy 
experience which received emphasis were history concepts relating to primary sources (71), 
multicultural perspectives (64), and critical evaluation (29). Teachers also made frequent 
references to the booklet "Building a History Curriculum n (31). While many items were deemed 
as contributing to new understanding, only specific topical knowledge and selected historical 
processes were aligned with changes in use. 
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While the interviews conducted at the end of the Academy's summer session provided 
information about the immediate impact of the program, information obtained from the follow up 
interviews contributed to our ability to assess how well the teachers would be able to sustain and 
extend the effects from the summer sessions* 

Those data suggested that the impact of the Academy was sustained, enriched and 
expanded* Academy participants were still actively engaged with the ideas presented and had 
appropriated many as well as imparted some. 

As before, Academy participants approached their experience generally from the 
perspective of "teacher". In particular, participants were about four times as likely to view and 
evaluate an experience in terms of its classroom application (414) as they were in terms of its 
relationship to historian scholarship (122) and about ten times more likely than to relate it to then: 
own personal interests (48). In contrast to the summer interviews, however, about half as many 
responses reflected an "historian" perspective^ 1 1/122). Tables 5 & 6 represent a summary of the 
impact of the Academy during the following school year. 

Table 5: Utility of Academy by Perspective for Twelve Case Study 
Participants During Follow-Up Interview 

FOLLOW-UP INTERVIEW 





Citation 
0) 


Understanding 
(2) 


Appropriating 
(3) 


Imparting 
(4) 


Rejection 
(5) 


Teacher 


69 


108 


109 


81 


47 


Historian 


11 


16 


48 


38 


9 


Individual 


6 


9 


17 


7 


9 
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Table 6: General Knowledge Shifts by Perspective for Twelve Case 
Study Participants During Follow-Up Interview 





New 

Knowledge 


FOLLOW-UP INTERVIEW 

Shift in Reinforcement 
Knowledge of Knowledge 


No 
Effect 
(Ql 


Teacher 


204 


76 


90 


44 


Historian 


48 


31 


34 


9 


Individual 


17 


11 


12 


8 



Again, in an effort to examine the various hot spots, various analyses detailing the impact 
of each element in terms of level of utilization and knowledge shift were carried out These are 
presented in Tables 7 and 8 respectively. 
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Table 8: The Relation of Knowledge Shift to Critical Item By 
Perspective During Follow-Up Interview 
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Model units - Betty, Nancy, et al. 


10 


o 


0 


o 


o 


n 


3 


0 2 


2 


o 


0 


R10 


Handouts 


2 


o 


0 


o 


o 


n 


1 


0 0 




o 


0 


R11 


Topic specific printed matter 


9 


1 


4 

1 


2 


0 


n 

*j 


2 


1 1 


0 


0 


0 


R12 


Top ^ specific other media 


4 


1 


0 


0 


0 


0 


0 


0 0 


0 


0 


0 


R13 


Historian 


0 


0 


0 


o 


3 


1 


2 


2 1 


5 


5 


5 


R14 


Portfolios 


9 


0 


3 


1 


o 


n 
\j 


6 


1 4 


2 


1 


1 


R15 


.Journal 


0 


0 


0 


4 


0 




1 


0 0 


1 


o 


0 


R16 


World history course 


4 


0 


1 


0 


0 


n 


0 


0 0 


1 


o 


0 


R17 


American history course 


2 


0 


0 


0 


0 


n 


1 


0 1 


0 


0 


0 


R18 


Theory and methods course 


1 


0 


0 


1 


0 


0 


0 


0 0 


0 


0 


0 


E01 


Guest lectures 


1 


0 


0 


0 


0 


0 


0 


0 0 


1 


0 


0 


E02 


Ohio Histoncal Society 


8 


1 


1 


0 


0 


0 


2 


1 0 


1 


0 


0 


E03 


Ohio State Library 


1 


0 


0 


1 


0 


1 


0 


0 0 


0 


0 


0 


E04 


Computer Demonstration 


1 


0 


0 


0 


0 


0 


0 


0 0 


4 


0 


0 


EOS 


Grade Level Small Group 


2 


0 


0 


0 


0 


0 


0 


0 0 


0 


0 


0 


E06 


Discussion Group 


0 


0 


0 


1 


0 


0 


0 


0 0 


1 


1 


1 


C01 


Primary Sources 


20 13 


0 


2 


1 


0 


12 


9 0 


0 


0 


0 


C02 


Critical Evaluation 


8 


4 


0 


7 


6 


c 


2 


6 1 


1 


1 


0 


C03 


Multicultural Perspective 


7 


5 


0 


3 


3 


0 


10 


3 0 


0 


0 


0 


C04 


Posthoie Approach 


1 


0 


0 


0 


0 


0 


8 


0 0 


2 


0 


0 


COS 


Significant Individual 


7 


7 


0 


0 


0 


0 


0 


0 0 


0 


0 


0 



As our analysis of the follow up interview indicates, participants were more likely to refer 
to ideas, resources, and strategies (attributed to the summer academy) they had "appropriated" and 
"imparted" (174,126) than they were during the earlier interview (199,0). Similar to the earlier 
interviews, participants continue to refer to information and ideas they had acquired since the 
summer session as "new". Participants were about twice as likely to consider an item as "new" 
(269) as they were to view it as a "change" (118) or as "reinforcement" (136) to their thinking. In 
comparison with the impact on knowledge associated with the summer session (528, 115, 132) the 
amount of information perceived as "new" fell by half, while the other descriptions remained 
constant In contrast to the summer, the type of ideas that the participants deemed salient did shift/, 
participants were about as likely to refer to curriculum (165) issues as they were to resources they 
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had collected (164). Additionally, teachers* responses suggested they emphasized issues of 
historical processes (134) over specific content knowledge (58), 

In terms of whether the effects of specific elements of the Academy were still active, the 
responses to the previously listed critical elements appears to have remained stable, except for 
references to print and nonprint content materials which decreased (159) (22), Consistent with the 
earlier interview, the participants' responses emphasized items and events which related to history 
concepts such as "primary source documents' 1 (57), "critical evaluation" (31), and "multicultural 
perspectives" (32) in describing their experiences since the summer Academy, Further, their 
responses suggest that participants were more likely to have "imparted" ideas about these concepts 
(especially "primary source documents") than they were other critical elements. The text, 
"Building a History Curriculum" (31) was also emphasized during these follow up conferences. 
Participants also frequently referred to their personal historian (8), particularly when adopting an 
"individual" perspective. 

Cross Validation 

As a means of cross-validating the trends emerging from the analyses of the 12 case 
studies, a short follow-up survey was distributed to all Academy teachers. Of the forty 
questionnaires mailed, eighteen were returned. The respondents included eight elementary school 
teachers, eight middle school teachers, and two high school teachers. The items on the survey 
corresponded to the tenets and coding emerging from the one to one interview with our subsample 
of twelve teachers. 

For the survey, teachers were asked to comment on 25 separate features of the Academy, 
Each feature received two responses — 

• one aimed at describing the feature's on the respondent's thinking 
(response choices were "no effect on my thinking", "new idea", "reinforced my 
own ideas", "changed my thinking") and 

• one aimed at describing the feature's utility (response choices were "not useful", 
"interesting, but not useful", "will use", "using or have used"), 
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Table 9: Frequency of teacher responses to cross-validation survey items 



1 NO EFFECT ON MY TANKING 


10 


NEW IOEA OR NFORMATON 


3 


RENFORCED MY OWN IDEAS 


3 


CHANGED MY THINKING 


2 NOT USEFUL 


4 


INTERESTING, BUT NOT USEFUL 


3 


WILL USE 


• 


USING OR HAVE USED 


World history text book 


2 NO EFFECT ON MY THNKING 


7 


NEW IDEA OR NFORMATON 


4 


RENFORCED MY OWN IDEAS 


0 


CHANGED MY THINKING 


2 NOT USEFUL 


2 


INTERESTING, BUT NOT USEFUL 


5 


WILL USE 


• 


USING OR HAVE USED 


American history lectures 


0 NO EFFECT ON MY THINKING 


5 


NEW IDEA OR NFORMATON 


4 


REINFORCED MY OWN DBAS 


9 


CHANGED MY THINKING 


0 NOT USEFUL 


1 


INTERESTING, BUT NOT USEFUL 


7 


WILL USE 


• 


USING OR HAVE USED 


American history text 


book 










2 NO EFFECT ON MY THNKING 


7 


NEW IDEA OR INFORMATION 


7 


RENFORCED MY OWN DEAS 


2 


CHANGED MY THINKING 


1 NOT USEFUL 


2 


INTERESTING, BUT NOT USEFUL 


7 


WILL USE 


7 


USING OR HAVE USED 


Guest lectures 


1 NO EFFECT ON MY THINKING 


5 


NEW IDEA OR NFORMATON 


3 


REN FORCED MY OWN DEAS 


7 


CHANGED MY THINKING 


0 NOT USEFUL 


4 


INTERESTING, BUT NOT USEFUL 


5 


WILL USE 


7 


USING OR HAVE USED 



After the Fact 



1 NO EFFECT ON MY THINKING 


7 NEW IDEA OR NFORMATON 


1 RENFORCED MY OWN DEAS 


• CHANGED MY THINKING 


1 NOT USEFUL 


9 INTERESTING. BUT NOT USEFUL 


7 WILL USE 


5 USING OR HAVE USED 



Historical Literacy 



2 NO EFFECT ON MY THINKING 


5 NEW IDEA OR N FORMATION 


1 RENFORCED MY OWN DEAS 


7 CHANGED MY THINKING 


1 NOT USEFUL 


5 INTERESTING, BUT NOT USEFUL 


7 WILL USE 


5 USING OR HAVE USED 



Narrative of the Life of Frederick Douglass 



2 NO EFFECT ON MY THNKING 


2 NEW IDEA OR ft FORMATION 


4 RENFORCED MY OWN DEAS 


1 0 CHANGED MY THINKING 


0 NOT USEFUL 


3 INTERESTING. BUT NOT USEFUL 


7 WILL USE 


7 USING OR HAVE USED 



Building a History Curriculum 



1 NO EFFECT ON MY THNKING 


6 NEW IDEA OR NFORMATON 


3 RENFORCED MY OWN DEAS 


6 CHANGED MY THINKING 


1 NOT USEFUL 


1 INTERESTING. BUT NOT USEFUL 


8 WILL USE 


7 USING OR HAVE USED 



Theory and methods course 



1 NO EFFECT ON MY THINKING 


5 NEW IDEA OR N FORMATION 


3 RENFORCED MY OWN DEAS 


6 CHANGED MY THINKING 


1 NOT USEFUL 


0 INTERESTING. BUT NOT USEFUL 


7 WILL USE 


7 USING OR HAVE USED 



Your historian 



5 NO EFFECT ON MY THNKiNH 


4 NEW IDEA OR NFORMATON 


1 RENFORCED MY OWN DEAS 


4 CHANGED MY THINKING 


4 NOT USEFUL 


3 INTERESTING. BUT NOT USEFUL 


• WILL USE 


2 USING OR HAVE USED 



Small group discussions 



2 NO EFFECT ON MY THNKING 


3 NEW IOEA OR NFORMATON 


2 RENFORCED MY OWN DEAS 


7 CHANGED MY THINKING 


1 NOT USEFUL 


2 INTERESTING, BUT NOT USEFUL 


4 WILL USE 


7 USING OR HAVE USED 


Grade level discussions 


1 NO EFFECT ON MY THNKING 


3 NEW IDEA OR NFORMATON 


4 RENFORCED MY OWN DEAS 


• CHANGED MY THINKING 


1 NOT USEFUL 


0 INTERESTING, BUT NOT USEFUL 


4 WILL USE 


9 USING OR HAVE USED 


informal discussions with other teachers 


0 NO EFFECT ON MY TH NKING 


4 NEW IOEA OR NFORMATON 


5 RENFORCED MY OWN DEAS 


• CHANGED MY THINKING 


0 NOT USEFUL 


1 INTERESTING, BUT NOT USEFUL 


5 WILL USE 


10 USING OR HAVE USED 
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Table 9: (continued) 
informal discussions with mentor teachers and others 



0 NO EFFECT ON MY THNKJNG 


4 NEWDEAORNFORMATON 


3 RENFORCED MYOVWDEA8 


7 CHANGED MY THINKING 


0 NOT USEFUL 


3 INTERESTING, BUT NOT USEFUL 


4 WU USE 


10 USING OR HAVE USED 


Ohio Historical Soclsty 


3 NO EFFECT ON MY THNKJNG 


S NEW IDEA OR NFORMATJON 


4 RENFORCED MYOVW DEAS 


3 CHANGED MY THINKING 


3 NOT USEFUL 


2 INTERESTING. BUT NOT USEFUL 


7 WILL USE 


S USING OR HAVE USED 


Your research project 


1 NO EFFECT ON MY THNKING 


4 NEW IDEA OR NFORMATDN 


4 RENFORCED MY OWN DEAS 


S CHANGED MY THINKING 


0 NOT USEFUL 


1 INTERESTING, BUT NOT USEFUL 


2 WILL USE 


15 USING OR HAVE USED 


Your teaching unlt(s) 


1 NO EFFECT ON MY THNKJNG 


4 NEW OEA OR INFORMATION 


S REWFORCED MY OWN DEAS 


S CHANGED MY THINKING 


0 NOT USEFUL 


1 INTERESTING, BUT NOT USEFUL 


3 WILL USE 


14 USING OR HAVE U3ED 


Your portfolio 


4 NO EFFECT ON MY THINKING 


S NEW ttEA CONFORMATION 


2 RENFORCED MYOVW DEAS 


6 CHANGED MY THINKING 


3 NOT USEFUL 


2 INTERESTING, BUT NOT USEFUL 


4 WILL USE 


S USING OR HAVE USED 



4 NO EFFECT ON MY THNKING 


3 NEW IDEA CONFORMATION 


6 RENFORCED MY OWN DEAS 


5 CHANGED MY THINKING 


4 NOT USEFUL 


S INTERESTING, BUT NOT USEFUL 


3 WILL USE 


5 USING OR HAVE USED 



"Post Hole" approach 



22 NO EFFECT ON MY THNKJNG 


3 NEW DEA OR N FORMATION 


2 RENFORCED MY OWN DEAS 


4 CHANGED MY THINKING 


0 NOT USEFUL 


2 INTERESTING. BUT NOT USEFUL 


3 WILL USE 


7 USING OR HAVE USED 


Primary sources 


0 NO EFFECT ON MY THNKING 


5 NEW IDEA OR NFORMAHON 


3 RENFORCED MY OWN DEAS 


9 CHANGED MY THINKING 


0 NOT USEFUL 


1 NTE RESTING, BUT NOT USEFUL 


S WILL USE 


12 USING OR HAVE USED 


Multicultural perspective 


0 NO EFFECT ON MY THNKJNG I 0 NEW DEA OR NFORMATON 


10 RENFORCED MY OWN DEAS 


7 CHANGED MY THINKING 


0 NOT USEFUL | 0 INTERESTING, BUT NOT USEFUL 


4 WILL USE 


14 USING OR HAVE USED 


Multiple perspectives 


1 NO EFFECT ON MY THNKING 


4 NEW IDEA OR N FORMAT ON 


5 RENFORCED MY OWN DEAS 


7 CHANGED MY THINKING 


1 NOT USEFUL 


1 INTERESTING, BUT NOT USEFUL 


S WILL USE 


10 USING OR HAVE USED 


Critical thinker 


1 NO EFFECT ON MY THNKING 


2 NEW DEA OR NFORMATON 


S RENFORCED MY OWN DEAS 


S CHANGED MY THINKING 


1 NOT USEFUL 


1 INTERESTING. BUT NOT USEFUL 


2 WILL USE 


14 USING OR HAVE USED 
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Table 9 includes the results for all the teachers item by item. 

In general, the data from the survey was consistent with the interview data from the 
subsample of twelve teachers. As with the case study findings, nearly all the features were 
perceived by teachers as having an effect on their thinking and also having some degree of 
usefulness. Some items drew distinct responses (that is, teachers seemed to cluster at the same 
response) while other items drew responses which were less distinct (responses were varied). 

Among specific features, the idea of "primary sources" appeared to be most powerful. 
This was also the case with the interview data- Nearly all teachers felt this concept was useful, and 
about have the respondents reported that this idea changed their thinking in some way. The notion 
of "multiple perspectives" received a similarly strong response. Teachers also generally viewed the 
research project and the teaching unit as a powerful aspect to the Academy experience. Nearly all 
respondents reported that these projects affected their thinking in some way. The majority of 
teachers also reported that they were using or had already used these projects in some way. 

Other highly-effective features included less structured aspects of the Academy, in 
particular formal and informal small group "grade level & small group discussions", as well as 
"informal discussions with mentor teachers". The portfolio feature was generally viewed as useful, 
and nearly a third of the teachers stated that ideas about portfolios were new to them. 

Some features viewed as highly-effective were also perceived as reinforcing (rather than 
informing or changing) teachers' thinking. These features included "critical thinking" and 
"multicultural perspectives." 

However, not all features were perceived as having great utility. For example, teachers 
were evenly divided in the ways they viewed their relationship with a professional historian, with 
the most frequent response being "no effect, not useful." Other features simply lacked definite 
impact. For example, teachers were split in the way they viewed journal-writing. The concept of 
the "post hole" approach to history curriculum appeared to lack definition. Nearly half the teachers 
failed to respond to this feature in any way. 
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Teachers differed in their responses to the three courses presented in the Academy -World 
History, American History, and Theory and Methods. While they generally viewed all three 
courses as useful, the American History lectures seemed to be perceived as the most useful, 
followed by the Theory and Methods activities. The World History course, while viewed as 
having introduced more new ideas, seemed less applicable to their classroom practice. 

Taken together and consistent with our interview data, the survey suggests that teachers are 
motivated by practicality and that features which were perceived as having changed their thinking 
were also generally those which they were able to use. For example, teachers commented that 
while the information they were learning may be new, and even interesting, they could not find a 
way to incorporate this experience into their curricula. In contrast, the experience of assembling 
research projects and teaching units may have been regarded as highly effective (at changing their 
thinking) because these experiences were easily adapted to their practice. 

SUMMARY AND DISCUSSION 

Overall, the Academy appears to have achieved its purpose of creating a momentum for 
history education. The data clearly illustrates ways in which the Academy experience has become 
on-going and generative. The Academy affected changes in teachers' views of themselves (as 
professionals, historians and history teachers), their knowledge and practice. Moreover, these 
changes went beyond reinforcement of existing ideas and improvements upon existing practices. It 
led to a process of achieving new understandings and shifts and extended to the imparting of these 
ideas to others. The transfer that occurred was quite extensive. The data presents patterns in 
which ideas were effectively introduced and adopted by Academy participants who in turn imparted 
many of their ideas to their students and to their colleagues. Further, teachers were able to transfer 
certain concepts from the discipline of history and apply those to other areas of their curriculum 

The Academy successfully customized its approach to meet the diverse backgrounds, 
needs, assumptions, and individual differences of the participants. The overall collaborative and 
collegia! context of the Academy provided a model learning structure. Within this structure 
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teachers were assisted in defining and designing investigations and teaching materials that fit the 
requirements of their classroom needs and satisfied their individual curiosities. In addition, the 
organizers of the Academy chose a multifaceted and collaborative approach. They connected 
history or being an historian with teaching history or history education. The convergence of the 
discipline of history with practice appears to have been responsible for achieving some of the 
aforementioned goals. Teachers were treated with integrity. They were assisted in becoming 
historians and history educators by historians and history educators. Multiple resources gave 
teachers access to new ideas, as well as mobilized or equipped them to implement new practices. 
Multiple formats of presentation afforded the critically engagement with ideas, consider multiple 
perspectives and explore new possibilities. These included lectures, formal group discussions, 
and informal talks with mentors and colleagues. Also, the Academy was organized around 
powerful and recurring organizing themes. These ihemes were tied to historical processes and 
were generally accessible to the teachers, who in turn, appeared to have appropriated these themes 
as the framework for their own classrooms. 

This evaluation procedures were also unique — especially, the use of portfolio evaluation 
procedures as a way of pursuing a grounded evaluation of the influence of a project upon the 
intellectual journey and practice of teachers. The data emerging from these portfolios were 
complex and varied providing a rich and detailed picture of individual teachers changes as well as 
the factors or elements which contributed to those changes. Without compromising the integrity of 
the assessment, these data and data collection procedures were pursued collaboratively. Without 
diminishing the information or feedback that was needed, these data were examined on their own 
terms. Indeed, these data seem more trustworthy and richer than more traditional means of 
assessment. 

In closing, the Academy was remarkably successful in an attempt to change participants 1 
perspectives about the discipline of history and about themselves as tet, tiers. This success can be 
traced to the Academy's concerted effort to transform teachers into historians by enabling them to 
do history. This aspect of the Academy was clearly, highly effective. Teachers used bonafide 
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methods of historic* 1 inquiry to collect data and to conduct original research. This was also an 
effective means to refocus teachers' perceptions of themselves (as teachers, historians, and 
individual learners). And, to a great extent, after "doing history" most participants changed their 
view of the discipline and the materials and methods they typically used to teach it. 

This opportunity to become historians had a powerful effect on teacher's classroom 
strategies. This is particularly well-supported in the number of references made to "primary source 
documents" and "critical evaluation" in the context of classroom applications. Teachers explained 
that a main reason for applying to the History Academy was to learn ways to make history come 
alive for their students. By appropriating and adapting ideas learned during the Academy, it 
appears they have found these perspectives and tools as effective ways to do just that 

The evaluation procedures should also be heralded. They represent a shift in evaluation 
toward practices which begin to live up to how history is done and should be evaluated. 
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Summary of Key Terms 



In looking at the graphs 
terms and their definitions may 

PERSPECTIVE 

Teacher: 

Historian: 

Individual: 

UNDERSTANDING 

New Knowledge: 
SNfts in Knowtodfp: 

Reinforcement 
No Effect 
INTENSITY/UTILITY 
Rejection: 

Citing: 

Understanding: 

Appropriating: 

imparting: 

TOPICS 

Resources: 

Teaching Strategies: 
Curriculum: 

Knowledge Base: 
Historical Processes: 

Cottegiality: 
Technology: 



and data for any of the participants, the following summary of 
be helpful. 

References to themes commonly attributed to teachers- instruction, teaching 
resources, management of children, ate. 

References to themes commonly attributed to historians - resaarch processes, 
interpretative techniques, resources, eta 

References to themes cornmonly attributed to individuals - personal values and goals, 
opinions and behaviors, ate. 

New information regarding topics or procedures 

Information conflicting with previous understandings and resulting in shifts, 
derivations, or eisborsbons 

Information supporting previousrv-heki beliefs and behaviors 
Information having no effect on thinking 

Information Individual either ignored or perceived as inapplicable 
Information listed without elaboration 
Information explained and interpreted 
Information adapted to individual needs 
Information transferred to students or colleagues 

Printed matter, key places, and people perceived as sources of pedagogical or 
historical information 

Specific classroom activities, teaching methods 

Pedagogical issues pertaining to scope and sequence of instruction, assessment, 
educational policy 

Historical information pertaining to specific units of research or instruction, 
Eplstemologlcal issues pertaining to trends in the research and Interpretation of 
historical phenomena 

Professional interaction with Academy participants, school colleagues, and historians 
Use of computers and other implements of technology to assist research, instruction, or 
personal development 
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KEY ELEMENTS (HOT SPOTS) 



RESOURCES 


R01 


Building a History Curriculum 


R02 


History Matters 


R03 


American History Textbook 


H04 


World History Textbook 


R05 


After the Fact 


R06 


Historical Literacy 


R07 


Narrative of Frederick Douglass 


R08 


Lecture notes 


R00 


Model units - Betty, Nancy, et ai. 


R10 


Handouts 


R11 


Topic specific printed matter 


R12 


Topic specific other media 


R13 


Historian 


R14 


Portfolios 


R15 


Journal 


R16 


World history course 


R17 


American history course 


R18 


1 1 I^KJf J mt K4 (HffU VUI lA^JI W 


EVENTS 




E01 


Guest lectures 


E02 


Ohio Historical Society 


E03 


Ohio State Library 


E04 


Computer Demonstration 


EOS 


Grade Level Small Group 


E06 


Discussion Group 


CONCEPTS 


C01 


Primary Sources 


C02 


Critical Evaluation 


C03 


Multicultural Perspective 


C04 


Poe thole Approach 


C05 


Significant Individual 



Booklet prepared by the Bradley Commission containing the 
Habits of the Mind and guidelines for teaching history in schools 
Publication prepared by the National Council for History 
Education which contains ideas, notes, and news about history. 
Selections from this book were assigned as lead-ins for lectures 
and was meant to also serve as a reference for further study. 
Selections from this book were assigned as lead-ins for lectures 
and was meant to also serve as a reference for further study. 
Selections from this book were assigned as lead-ins for lectures 
and as an introduction to the art of historical detection. 
Selections from this book were assigned as lead-ins for lectures. 
Selections of this book were assigned as a lead-in to lecture. 
Participants' recording of information from lectures 
Mentor teachers presented model units in the theory and 
methods course demonstrating practical application. 
Copies of materials distributed ID participant! as lead-ins for 
lectures to come, units presented in class, information 
presented by guest speakers. 

Books, articles, units, newspapers, magazines which pertained 
to the participants' research project or unit. 
Records, tapes, videotapes, artifacts, or persons relating to 
their research project or unit. 

Participants were to choose an historian from a list. This 

historian was to help them with their project topic and unit by 

providing Information, bibliographies, etc. 

Alternative method of evaluation presented to the participants, 

as welt as the physical materials gathered by the participants to 

represent their experience in the Academy. 

Participants were to reflect in writing daily during the Academy 

and periodically during the following school year. 

Books, lectures, speakers, and other materials used in 

presenting information on countries other than the US. 

Books, lectures, speakers, and other materials used in 

presenting information on the US and its development. 

Books lectures, speakers and other materials used in 

presenting pedagogy and practical application of historical 

education concepts. 

Individuals speaking to the participants other than the Academy 
personnel. 

A trip to the Ohio Village and Ohio Historical Society which 

Introduced participants to their library and resources 

A trip to The Ohio State University main library in which they 

were Introduced and tutored in computer system and how to 

locate topics, books and related materials. 

Presentations of various software and hardware for educational 

use. 

Participants were assigned to meet with one member of 
Academy personnel and other participants with similar teaching 
assignments for the purpose of more In depth discussion of 
presented materials, as well as sharing of methods and 
materials that have worked for them in the past 
Participants were assigned to meet with one member of 
Academy personnel and other participants regardless of grade 
level for the purpose of more in depth discussion of topics. 

Information purporting to be first-hand, eye-witness accounts o! 
historical events. Written documents, oral histories, paintings, 
photographs, etc. 

Question information source validity, circumstances, and 
interpretations. Read widely and critically, ask probing 
questions. 

Accepts the proposition that interpretation of historical events 
and trends is influenced by the presenting dominant culture. 
A method of organizing instruction or exploration around key 
aspects of period. 

Reference to the Importance of individuals who have made a 
difference in history and the significance of personal character 
for both good and ill. 
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APPENDIX B 
A sample case study 



ERIC 



36 



Portfolio Evaluation As History 9/7/93 



Tlemey, Wile, Moss, et al. — Page 35 



Rachael 

General Background 

Rachael is a seventeen-year veteran, currently assigned to grade four at a suburban 
elementary school She described her background in history as two courses on the teaching of 
social studies and history. At the time of this Academy, educational level was Bachelor of Arts 
plus thirty hours. Rachael attended the Academy with her teaching partner, another grade four 
teacher from her school 

In her application letter, Rachael identified her expectations and goals for the History 
Academy as enabling her to "develop a teaching area for my fourth graders that would bring to life 
a history of Ohio that most of the children's grandparents and even some of their parents lived 
through. Rachael attributed her interest in history to her parents, two teachers and self-appointed 
history buffs." 

The Academy Experience 

As indicated by the first three graphs for the summer exit interview, the majority of 
Rachael's responses reflected a teacher perspective. In only two responses did Rachael refer to 
ways in which the Academy influenced her as an individual. The majority of responses, regardless 
of perspective, were reference to specific resources and instructional materials. These resources 
tended to be described as "new" to Rachael and she appeared to have incorporated these resources 
into her own repertoire at the level of citation or understanding. Nearly half the elements referred 
to topic specific printed matter. Among historical concepts, Rachael cited primary sources as an 
influential concept. 

As indicated by the second three graphs for the final follow-up interview, Rachael's 
comments during the follow-up conference were dominated again by responses which 
characterized a teacher perspective. Her interpretations tended to describe the utility of various 
resource materials, teaching strategies, curriculum, and the acquisition of new knowledge about 
her content topic — Ohio history. However, Rachael increased the amount of references which 
reflected the perspective of historian. The influence on Rachael as an individual appears minimal 
from these comments. Rachael comments centered around resources, curricular issues, teaching 
strategies, and on expanding her knowledge base of topical information. Comments on historical 
concepts and processes were almost as frequent as references which suggested a teacher 
perspective. Most of Rachael's comments during the follow-up conference referred to knowledge 
shifts. These new or shifting ideas were generally appropriated and imparted to students. Critical 
features of the Academy appeared to be portfolios, the model lessons presented by the Academy 
faculty, and the resource Building a History Curriculum. Rachael had also taken on administrative 
duties in addition to her grade four assignment. During the follow-up interview, Rachael spoke 
enthusiastically and explicitly about the impact of the Academy experience on her teaching and her 
students' learning. Rachael explains that "because we are kind of limited with Ohio history I've 
always before just kind of focused right on Ohio. Now I find I'm talking more about European 
history and history that happened before Ohio. Now I find that I'm looking more at relationships. 
So I think it was really a good experience." 
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If you can hear the 
beat of drums from 
other times... 

...from other places; 



% 



..if you believe knowledge 
of history is essential for sound 
judgements in personal and public life; 



..if you think the study of history should 
begin in elementary school ...but never end; 

...then we need you in the National Council for History 
Education! 



As part of the Council you will: 

• receive your monthly newsletter, 
History Matters!, via first class mail, 
which will keep you up-to-date on 
history activities and opportunities 
around the country 

• have a two-way link (via phone, mail, 
and e-mail) to others who value history 
and are working to promote it in and 
out of school 

• support the active advocacy of history 
in the school curriculum through the 
Council Speakers' Bureau; school staff 
development and teacher inservice 



workshops; public testimony before 
hearings on state and local curriculum; 
and review of curriculum documents 
and guidelines 

• have access (at discount prices) to 
books and reports published by the 
Bradley Commission on History and to 
all new publications and materials 
developed by the Council. 

Be on the "front lines* of the battle to 
promote history in school and society. 
Please join us because, as Uncle Sam 
used to say, "We Need You!" 



Please enroll me as a member of NCHE. 



YES, I want to support the cause of history education. Here's my contribution. Please send my 
New Member Packet, and mail my monthly newsletter, History Matters! via First Class Mail. 

0$25 O$50 O$100 O Other$ 

Name Return to: 



,n8tltution National Council for 

Address History Education 

City. State ZIP 26915 Westwood Rd., 

' , ; Suite B-2 

Phone: ( ) Westlake, OH 44145 

Electronic Mail Address: 



^' lr ?temet, Bitnet, CompuServe, America OnLine, Freenet, etc.) 
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Evaluation of the History Academy for Ohio Teachers 
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National Council for History Education, Inc. 
26915 Westwood Rd., Suite B-2 
Westwood, OH 44145 
Ph: 216-835-1776 
FAX: 216-835-1295 
email (Internet): ae515@cleveland.freenet.edu 
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